[Python APIView] update supported py versions + related fixes by swathipil · Pull Request #14708 · Azure/azure-sdk-tools

swathipil · 2026-03-25T00:29:16Z

Adding support for Python versions 3.9-3.14 (since only 3.10 was working) to available for use by the azpysdk CLI command for api.md generation.

TODO:

add + test 3.14
add more tests to cover all possible syntax and run across all py versions

Copilot

Pull request overview

Updates the Python APIView stub generator and its test package to reflect newer supported Python versions (dropping older runtimes and adding newer ones), along with parsing/test adjustments to keep output stable across Python 3.9–3.13.

Changes:

Bump apiview-stub-generator to 0.3.28 and update supported Python versions in packaging metadata and changelog.
Expand CI to run tox-based tests across Python 3.9–3.13 and adjust tests/fixtures for version-specific behavior.
Improve class/function parsing to better preserve source-level base class/keyword representations (e.g., metaclass) and normalize TypedDict/forward-ref rendering.

Reviewed changes

Copilot reviewed 17 out of 20 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
packages/python-packages/apiview-stub-generator/tests/dataclass_parsing_test.py	Makes dataclass expectations version-aware and skips KW_ONLY test on <3.10.
packages/python-packages/apiview-stub-generator/tests/class_parsing_test.py	Updates expected TypedDict/Enum output and adds regression tests for forward-ref bases and EnumMeta naming.
packages/python-packages/apiview-stub-generator/tests/azure_tests/token_files/azure-eventhub-checkpointstoreblob_python.json	Updates stored token output metadata for parser 0.3.28.
packages/python-packages/apiview-stub-generator/tests/azure_tests/token_files/azure-eventhub-checkpointstoreblob-aio_python.json	Updates stored token output metadata for parser 0.3.28.
packages/python-packages/apiview-stub-generator/tests/apiview_test.py	Adjusts diagnostic count expectation based on Python version behavior.
packages/python-packages/apiview-stub-generator/setup.py	Updates `python_requires` and classifiers to newer Python versions.
packages/python-packages/apiview-stub-generator/ci.yml	Runs test suite under Python 3.9–3.13 in CI.
packages/python-packages/apiview-stub-generator/apistub/nodes/_function_node.py	Guards against non-function AST nodes from `astroid.extract_node`.
packages/python-packages/apiview-stub-generator/apistub/nodes/_class_node.py	Adds AST-based base/keyword extraction, TypedDict normalization, and annotation handling consistency.
packages/python-packages/apiview-stub-generator/apistub/nodes/_base_node.py	Fixes Optional wrapping detection differences in Python 3.9 string representations.
packages/python-packages/apiview-stub-generator/apistub/_version.py	Bumps version to 0.3.28.
packages/python-packages/apiview-stub-generator/CHANGELOG.md	Documents 0.3.28 changes and version support updates.
packages/python-packages/apistubgentest/setup.py	Updates supported Python classifiers and `python_requires`.
packages/python-packages/apistubgentest/apistubgentest/models/_models.py	Adds new model for forward-ref base testing and updates typing syntax for 3.9 compatibility.
packages/python-packages/apistubgentest/apistubgentest/models/_mixin.py	Updates typing syntax for 3.9 compatibility.
packages/python-packages/apistubgentest/apistubgentest/models/_dataclasses.py	Gates KW_ONLY usage to 3.10+ and imports accordingly.
packages/python-packages/apistubgentest/apistubgentest/models/init.py	Conditionally imports KW_ONLY dataclass and adds new exports.

Comments suppressed due to low confidence (1)

packages/python-packages/apistubgentest/apistubgentest/models/init.py:59

__all__ includes "DataClassWithKeywordOnly" unconditionally, but on Python < 3.10 that symbol is never imported/defined (it's gated behind if sys.version_info >= (3, 10)). This will break from apistubgentest.models import * on 3.9 with an AttributeError. Consider only adding this name to __all__ when running on 3.10+, or always defining it (e.g., via a safe import/placeholder) so __all__ is accurate for each supported version.

    "DataClassSimple",
    "DataClassWithFields",
    "DataClassDynamic",
    "DataClassWithKeywordOnly",
    "DataClassWithPostInit",

packages/python-packages/apiview-stub-generator/apistub/nodes/_class_node.py

…ithub.com/swathipil/azure-sdk-tools into swathipil/pyapi/update-supported-versions

Copilot

Pull request overview

Copilot reviewed 21 out of 26 changed files in this pull request and generated 5 comments.

Copilot · 2026-04-03T17:40:13Z

packages/python-packages/apiview-stub-generator/apistub/nodes/_pylint_parser.py

+    @classmethod
+    def _normalize_namespace_inits(cls, path):
+        """Replace namespace azure/__init__.py files with empty files.
+
+        Packages distributed as sdist or source include a namespace-package
+        ``azure/__init__.py`` that extends ``__path__`` via ``pkgutil``.  When
+        pylint / astroid sees that file it merges all installed azure sub-packages
+        into a single namespace, which breaks decorator name resolution and causes
+        the azure-sdk guidelines checker to silently produce no diagnostics.
+        Overwriting those files with empty ones matches the behaviour of the WHL
+        variant (which never carries a non-trivial ``azure/__init__.py``).
+        """
+        for root, dirs, files in os.walk(path):
+            basename = os.path.basename(root)
+            if basename == "azure" and "__init__.py" in files:
+                init_path = os.path.join(root, "__init__.py")
+                try:
+                    with open(init_path, "r") as f:
+                        content = f.read()
+                    if "extend_path" in content or "pkgutil" in content:
+                        with open(init_path, "w") as f:
+                            pass  # overwrite with empty file
+                except Exception:
+                    pass
+                # No need to recurse deeper once we've found azure/
+                dirs.clear()
+


_normalize_namespace_inits() overwrites azure/init.py in-place under the provided path. If apistubgen is pointed at a source checkout (directory pkg_path), this will permanently mutate the user’s files and can leave the tree in a modified state even when parsing fails. Consider running pylint against a temporary copy of the package tree, or writing the original init.py contents back after pylint finishes (ideally in a finally block), and avoid swallowing exceptions so failures are visible.

Copilot · 2026-04-03T17:40:13Z

packages/python-packages/apiview-stub-generator/apistub/nodes/_pylint_parser.py

+        try:
+            raw_messages = json.loads(result.stdout or "[]")
+        except json.JSONDecodeError:
+            logging.warning("pylint produced non-JSON output; treating as no messages")
+            raw_messages = []


parse(): subprocess.run() doesn’t check the pylint exit code and treats any non-JSON stdout as "no messages". This can silently drop all diagnostics when pylint fails/crashes or when plugins emit non-JSON output. Consider: (1) handling non-zero returncode as an error (log stderr/stdout), and (2) only falling back to [] when you’re confident stdout is empty but pylint otherwise succeeded, so failures are surfaced rather than masked.

Suggested change

try:

raw_messages = json.loads(result.stdout or "[]")

except json.JSONDecodeError:

logging.warning("pylint produced non-JSON output; treating as no messages")

raw_messages = []

if result.returncode != 0:

logging.error(

"pylint failed for %s with exit code %s. stdout: %r stderr: %r",

path,

result.returncode,

result.stdout,

result.stderr,

)

raise RuntimeError(f"pylint failed for {path} with exit code {result.returncode}")

stdout = result.stdout.strip()

if not stdout:

raw_messages = []

else:

try:

raw_messages = json.loads(stdout)

except json.JSONDecodeError as exc:

logging.error(

"pylint produced non-JSON output for %s. stdout: %r stderr: %r",

path,

result.stdout,

result.stderr,

)

raise RuntimeError(f"pylint produced invalid JSON output for {path}") from exc

Copilot · 2026-04-03T17:40:13Z

packages/python-packages/apiview-stub-generator/apistub/nodes/_class_node.py

            class_decorators = class_node.decorators.nodes
            self.decorators = [
-                f"@{x.as_string(preserve_quotes=True)}" for x in class_decorators
+                f"@{x.as_string()}" for x in class_decorators


_parse_decorators_from_class(): calling astroid nodes’ as_string() without preserve_quotes will strip quotes from decorator arguments because NodeNG.as_string is monkey-patched to remove quotes by default. This will mis-render decorators like @another_decorator("Test") as @another_decorator(Test). Suggest using as_string(preserve_quotes=True) here as well (consistent with FunctionNode) so decorator source is preserved accurately.

Suggested change

f"@{x.as_string()}" for x in class_decorators

f"@{x.as_string(preserve_quotes=True)}" for x in class_decorators

Copilot · 2026-04-03T17:40:13Z

packages/python-packages/apiview-stub-generator/apistub/nodes/_class_node.py

+            # Add keyword arguments (e.g., metaclass=..., total=...)
+            for idx, keyword in enumerate(self.class_keywords):
+                # Parse keyword as "arg=value"
+                if "=" in keyword:
+                    arg, value = keyword.split("=", 1)
+                    line.add_text(arg, has_suffix_space=False)
+                    line.add_punctuation("=", has_suffix_space=False)
+                    line.add_type(value, apiview=self.apiview, has_suffix_space=False)
+                    # Add comma between multiple keywords
+                    if idx < len(self.class_keywords) - 1:
+                        line.add_punctuation(",")


generate_tokens(): class keyword argument values are emitted via add_type(), which makes sense for metaclass=SomeMeta but misclassifies literal values like total=False (TypedDict), kw_only=True, etc. This affects token kinds/navigation and can make booleans look like type names. Consider rendering keyword values as literals when they’re not type-like (e.g., True/False/None/numbers/strings), and only using add_type() when the value is intended to be a type reference.

Copilot · 2026-04-03T17:40:14Z

packages/python-packages/apiview-stub-generator/apistub/nodes/_class_node.py

+            # For dynamically-created classes (e.g. make_dataclass) that have no source file,
+            # only include functions that themselves have a real source file. This filters out
+            # generated methods like __init__ that were synthesized by the dataclass machinery,
+            # while keeping user-defined methods (e.g. lambdas passed via namespace=).
+            try:
+                inspect.getsource(self.obj)
+            except (OSError, TypeError):
+                sourcefile = inspect.getsourcefile(func_obj)
+                if not sourcefile or sourcefile == "<string>":
+                    return False
+            return True


_should_include_function(): inspect.getsource(self.obj) is executed on every member function check, even though the result is only used to detect “class has no source”. This adds repeated filesystem reads for classes with many members. Consider computing/caching a boolean like self._class_has_source once in _inspect()/init and reusing it here.

swathipil added 7 commits March 18, 2026 08:51

add a separate method to configure environment from url

63a6170

Merge branch 'main' of https://github.com/Azure/azure-sdk-tools

e6c6295

Merge branch 'main' of https://github.com/Azure/azure-sdk-tools

dbe07c6

update supported versions

9dce12b

fix bug with class parent source code repr

83fc869

add class keywords

b1a5093

exclude private class keywords

6f52468

swathipil changed the title ~~Swathipil/pyapi/update supported versions~~ [Python APIView] update supported py versions + related fixes Mar 25, 2026

github-actions bot added the azsdk-cli Issues related to Azure/azure-sdk-tools::tools/azsdk-cli label Mar 25, 2026

swathipil force-pushed the swathipil/pyapi/update-supported-versions branch from a754161 to 6f52468 Compare March 25, 2026 00:39

swathipil added 5 commits March 24, 2026 17:40

restore tools/azsdk-cli from upstream/main

6f8906d

fix py3.12 issues

3bd677d

fix TypedDict format

72b3218

py 3.9 compatibility

e324f6e

Merge branch 'main' into swathipil/pyapi/update-supported-versions

4cc410c

swathipil requested a review from Copilot March 26, 2026 20:25

Copilot started reviewing on behalf of swathipil March 26, 2026 20:27 View session

Copilot AI reviewed Mar 26, 2026

View reviewed changes

packages/python-packages/apiview-stub-generator/apistub/nodes/_class_node.py Outdated Show resolved Hide resolved

packages/python-packages/apiview-stub-generator/apistub/nodes/_class_node.py Outdated Show resolved Hide resolved

swathipil added 5 commits March 26, 2026 17:32

copilot comments

ed7ee61

Merge branch 'swathipil/pyapi/update-supported-versions' of https://g…

a044fba

…ithub.com/swathipil/azure-sdk-tools into swathipil/pyapi/update-supported-versions

merge main

b440ad9

add 3.14

98b7d4a

add more tests

c52ac3a

swathipil requested a review from Copilot April 3, 2026 17:32

Copilot started reviewing on behalf of swathipil April 3, 2026 17:33 View session

Copilot AI reviewed Apr 3, 2026

View reviewed changes

handle future typing union annotations

2534bb1

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Python APIView] update supported py versions + related fixes#14708

[Python APIView] update supported py versions + related fixes#14708
swathipil wants to merge 18 commits intoAzure:mainfrom
swathipil:swathipil/pyapi/update-supported-versions

swathipil commented Mar 25, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 3, 2026

Uh oh!

Copilot AI Apr 3, 2026

Uh oh!

Copilot AI Apr 3, 2026

Uh oh!

Copilot AI Apr 3, 2026

Uh oh!

Copilot AI Apr 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

-        try:
-            raw_messages = json.loads(result.stdout or "[]")
-        except json.JSONDecodeError:
-            logging.warning("pylint produced non-JSON output; treating as no messages")
-            raw_messages = []
+        if result.returncode != 0:
+            logging.error(
+                "pylint failed for %s with exit code %s. stdout: %r stderr: %r",
+                path,
+                result.returncode,
+                result.stdout,
+                result.stderr,
+            )
+            raise RuntimeError(f"pylint failed for {path} with exit code {result.returncode}")
+        stdout = result.stdout.strip()
+        if not stdout:
+            raw_messages = []
+        else:
+            try:
+                raw_messages = json.loads(stdout)
+            except json.JSONDecodeError as exc:
+                logging.error(
+                    "pylint produced non-JSON output for %s. stdout: %r stderr: %r",
+                    path,
+                    result.stdout,
+                    result.stderr,
+                )
+                raise RuntimeError(f"pylint produced invalid JSON output for {path}") from exc

	f"@{x.as_string()}" for x in class_decorators
	f"@{x.as_string(preserve_quotes=True)}" for x in class_decorators

Conversation

swathipil commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

swathipil commented Mar 25, 2026 •

edited

Loading